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AASERT:  Early  scene  analysis:  Rapid  processing  of 

CONTOURS,  SURFACES,  AND  OBJECTS  IN  HUMAN  VISION 


Summary 

This  report  covers  the  AASERT  grant  which  is  a  companion  grant  to  F49620-94- 
1-0189  of  the  same  title.  Over  the  duration  of  the  grant,  support  and  training  were 
provided  for  5  different  graduate  students  and  7  undergraduates.  The  work  concentrated 
on  how  2-D  information  is  built  up  from  the  parallel  analysis  of  a  set  of  visual  attributes 
and  how  this  information  contacts  memory  in  order  to  construct  3-D  representations  of 
the  visual  scene.  We  were  specifically  interested  in  the  early  stages  of  these  processes 
which  may  operate  under  simplified  assumptions  in  order  to  gain  speed.  We  examined 
the  rapid  decomposition  of  image  values  into  object  features  (reflectance,  transparency, 
orientation,  3D  position)  and  illumination  features  (shadows,  shading,  highlights).  We 
also  evaluated  the  nature  of  the  representation  achieved  at  these  early  levels  and  whether 
the  output  of  each  of  the  many  early  stages  is  independently  available  to  higher  level 
processes  or  only  one  final  “result”  gets  passed  along.  Finally,  we  studied  the  initial 
contact  between  the  image  contours  and  memory  in  recognition. 


Research  Training  Avtivities 

Graduate  Students.  Peter  Tse  has  developed  outstanding  research  skills 
including  programming,  and  scientific  writing.  His  work  over  the  period  of  support  from 
AASERT  has  already  established  his  reputation  as  a  major  creative  contributor  to  vision 
research.  He  won  the  prestigeous  McDonnell-Pew  Fellowship  for  3  years  and  will  begin  a 
postdoc  at  the  Max  Planck  Institute  in  Tubingen  this  fall.  His  training  has  been  a  great 
success.  During  his  period  of  support,  James  Intriligator  developed  several  software 
techniques  which  are  used  by  many  in  our  lab  and  his  work  on  attention  has  been  very 
influential.  Alex  Holcombe  and  David  Whitney  have  become  strong  experimenters  and 
have  good  potential.  Stephen  Hau  dropped  out  of  graduate  school  after  one  semester. 

Undergraduates.  Our  interest  in  the  internal  representation  of  object  shape  has 
led  us  to  focus  on  simple  two  tone  representations  of  shadowed  images.  Several 
undergraduates  participated  in  this  project  as  artists  producing  stimuli  for  our  priming 
experiments.  They  were  introduced  to  various  aspects  of  computer  controlled  image 
manipulation  and  to  theories  of  visual  perception  and  visual  memory.  One  undergraduate 
(Yuri  Ostrovsky)  developed  new  software  for  us  on  the  Warping  function  of  our 
DataCube  image  processor.  He  was  trained  in  programming  techniques  for  video  pipeline 
processors.  Christopher  Dye  learned  about  programming  and  image  presentation  during 
his  summer  assistantship. 


Accomplishments  /  New  Findings 

Object  recognition:  positive  priming.  In  our  model,  recognition  starts  with  an 
initial,  crude  2-D  match  that  selects  a  “best”  prototype  to  explain  the  image  data.  This  is 
followed  by  more  sophisticated  3-D  analyses  to  complete  the  recognition  process.  Our 
first  experiment  showed  a  priming  effect  of  contours  in  recognition  even  though  the 
contours  alone  were  uniformative  for  the  task.  David  Whitney  has  repeated  our  earlier 
experiments  during  using  a  new  set  of  subjects. 
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Object  models  and  motion  perception.  Peter  Tse  has  developed  a  new  theory 
for  apparent  motion  that  relies  on  parsing  each  scene  into  objects  before  matching  takes 
place.  The  novel  aspect  of  the  work  is  that  the  shapes  in  the  first  frame  of  the  motion 
sequence  overlap  spatially  with  those  in  the  following  frame.  This  enables  Peter  to  test 
for  principles  of  shape  parsing  (continuity,  surface  similarity,  contiguity)  that  do  not 
come  into  play  in  standard  apparent  motion  where  the  shapes  do  not  overlap.  This  give  us 
a  new  tool  for  understanding  image  segmentation.  This  work  is  published  in  a  recent 
book  (Tse,  Cavanagh,  &  Nakayama,  1998). 

Theory  of  volume.  Peter  Tse  has  developed  a  new  theory  for  the  level  at  which 
objects  are  represented  in  understanding  visual  scenes.  This  work  is  exceptionally  novel 
and  important.  Rather  than  depending  on  relations  between  image  contours  or  inferring 
surfaces,  Peter  shows  that  the  underlying  mode  of  representation  is  one  of  volumes  or 
occupied  space.  Several  critical  demonstrations  show  that  his  formulation  accounts  for 
the  broad  range  of  image  interpretations  whereas  representations  of  objects  by  their 
contours  or  surfaces  fail.  In  a  very  productive  year,  he  has  been  able  to  publish  four 
papers  on  this  topic  (Tse  &  Albert,  1998;  Tse,  1998a,  1998b)  and  present  four  talks  (Tse, 
1997b;  Tse,  1998a;  Tse  1998b;  Tse  &  Albert,  1998). 

Attention  resolution.  While  supported  by  this  grant  James  Intrilgator  contributed 
to  landmark  work  on  the  resolution  of  attention  published  in  Nature  and  Trends  in 
Cognitive  Science.  He  showed  that  the  ultimate  limit  on  performance  is  not  set  by  the  fine 
detail  you  can  see  but  rather  by  the  very  coarse  access  you  have  to  that  detail. 

Time  perception.  Peter  Tse  and  James  Intriligator  were  also  able  to  demonstrate 
the  intuitive  notion  of  the  slowing  of  time  during  attention-grabbing  events  (like  a  car 
accident).  An  attention-grabbing,  odd  item  in  a  sequence  of  items  appeared  to  last  about 
50%  longer  than  than  its  real  duration. 


Personnel  supported 

Personnel  on  this  grant.  Graduate  students:  Peter  Tse  (3  years,  6  months),  David 

Whitney  (8  months),  Alex  Holcombe  ( 1  year),  James  Intriligator  (2  years),  Stephen  Hau 

(4  months).  Undergraduates:  Christopher  Dye  (summer  research  assistant),  Jonathan  Liu, 

Derek  McKee,  Jason  Gross,  Naima  Workman,  Sebastian  Conley  (artists),  and  Yuri 

Ostovsky  (programmer). 
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Interactions,  conference  papers  during  grant  period  supported  by  AASERT  grant 

*Tse,  P.,  Cavanagh,  P.  &  Nakayama,  K.  (1995).  Line  motion  occurs  after  surface  parsing. 
Investigative  Ophthalmology  &  Visual  Science,  36,  S417. 
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Helsinki. 
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Philadelphia. 

Holcombe,  A.  0.,  Tse,  P.,  Macknik,  S.,  Seiffert,  A.  E.,  &  Intriligator,  J.  (1998).  Wakes:  a 
new  motion-induced  brightness  illusion.  Investigative  Ophthalmology  &  Visual 
Science,  39,  S671 

Tse,  P.  (1998).  Volume  relatability,  volume  completion,  and  illusory  volumes. 

Investigative  Ophthalmology  &  Visual  Science,  39,  SI  132. 

Tse,  P  and  Albert,  M.  (1998).  Amodal  completion  in  the  absence  of  image  tangent 
discontinuities.  ECVP,  Oxford  University. 

Tse,  P.  (1998).  Volume  Completion.  Eastern  Psychological  Association,  February, 
Boston. 

Whitney,  D.  V.,  Murakami,  I.,  &  Cavanagh,  P.  (1998).  Motion  extrapolation  cannot 
account  for  apparent  position  offset  of  a  flashed  disk  relative  to  unpredictable  motion. 
Investigative  Ophthalmology  &  Visual  Science,  39,  SI 075 


New  discoveries,  inventions,  or  patent  disclosures 

Discoveries  are  reported  in  the  progress  section  above.  There  were  no  inventions 
or  patent  disclosures  during  the  grant  period. 

Honors  /  Awards 


None. 


